-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.Net Agents - Add Streaming support for OpenAIAssistantAgent
and AgentChat
#8175
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…#7130) ### Motivation and Context This PR migrates the `AzureOpenAIAudioToTextService` to Azure.AI.OpenAI SDK v2. ### Description 1. The existing `OpenAIAudioToTextExecutionSettings` class is copied to the new `Connectors.AzureOpenAI` project and renamed to `AzureOpenAIAudioToTextExecutionSettings` to represent prompt execution settings for the `AzureOpenAIAudioToTextService`. 2. The `OpenAIAudioToTextExecutionSettings.ResponseFormat` property type has changed from string to the `AudioTranscriptionFormat` enum, which is a breaking change that is tracked in the issue - https://github.com/microsoft/semantic-kernel/issues/70533. 3. The `ClientCore.AudioToText.cs` class is refactored to use the new `AzureOpenAIAudioToTextExecutionSettings` class and to handle the new type of `OpenAIAudioToTextExecutionSettings.ResponseFormat` property. 4. Service collection and kernel builder extension methods are added to register the service in the DI container. 5. Unit and integration tests are added as well.
…for format (#7141) ### Motivation and Context - Updates Execution Settings for `ResponseFormat` from `string` to `enum`. - Updates the Unit Tests to validate.
### Motivation, Context, and Description 1. The `ClientCore.DeploymentOrModelName` property is renamed to `ClientCore.DeploymentName` to reflect it's actual purpose. 2. The `AzureOpenAIPromptExecutionSettings` class is moved to the `Settings` folder to reside alongside prompt execution setting classes for other services.
…7148) ### Motivation, Context and Description This PR: 1. Copies OpenAIFileService and the other classes related to it to the Connectors.AzureOpenAI project before refactoring them to use the Azure.AI.OpenAI SDK v2. 2. Renames the classes and changes namespaces accordingly. 3. Adds/changes no functionality in any of those classes. 4. Temporarily removes the classes from the compilation process. They will be included back in the next PR. --------- Co-authored-by: Roger Barreto <[email protected]>
### Motivation and Context This PR does the following: 1. Migrates OpenAIChatCompletionService, ClientCore, and other model classes both use, to OpenAI SDK v2. 2. Updates ToolCallBehavior classes to return a list of functions and function choice. This change is required because the new SDK model requires both of those for the CompletionsOptions class creation and does not allow setting them after the class is already created, as it used to allow. 3. Adapts related unit tests to the API changes. 4. Added netstandard2.0 as target for the UnitTest project. 5. Rename `TryGetFunctionAndArguments` to `TryGetOpenAIFunctionAndArguments` to avoid clashing with AzureOpenAI
### Motivation and Context - Obsolescence of FileService.
…#7203) ### Motivation and Context As a way to enforce the Non Breaking principle this PR remove all avoidable breaking changes from the V2 migration, including additions and small improvements identified.
### Motivation and Context It was decided not to migrate existing `OpenAIFileService` to the Azure.AI.OpenAI SDK v2 and eventually deprecate it. ### Description This PR removes files related to the `OpenAIFileService` added by a previous PR.
### Motivation and Context Removing non existing V1 logging.
### Motivation and Context This PR aligns metadata names with those provided by the underlying Azure.AI.OpenAI library. Additionally, it adds a few unit tests to increase code coverage.
…oft/semantic-kernel into feature-connectors-openai
### Motivation, Context and Description The new Azure.AI.OpenAI V2 introduced timestamp granularities for the audio-to-text API. This PR removes the first attempt to expose the settings in execution settings for the service. The setting will be introduced later after the migration is over in scope in this task: #7213.
### Motivation, Context and Description The unnecessary breaking change that was introduced earlier, which changed the type of the `OpenAIAudioToTextExecutionSettings.ResponseFormat` and `AzureOpenAIAudioToTextExecutionSettings.ResponseFormat` properties, has been rolled back.
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 6, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 6, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 6, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 6, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 6, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 7, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 9, 2024
github-merge-queue bot
pushed a commit
that referenced
this pull request
Sep 9, 2024
…gentChat` (#8175) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Add streaming support to direct invocation of `OpenAIAssistantAgent` and also `AgentChat`: Fixes: #4833 Fixes: #5643 Integrates with existing streaming support for `ChatCompletionAgent`. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Streaming returns the expected streaming content types as well as providing whole messages in the provided history. Support both agent invocation (no-chat) and via `AgentChat`. Introduced: - `StreamingAnnotationContent` - `StreamingFileReferenceContent` Added samples and tests. ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone 😄 --------- Co-authored-by: SergeyMenshykh <[email protected]> Co-authored-by: Roger Barreto <[email protected]> Co-authored-by: SergeyMenshykh <[email protected]> Co-authored-by: Roger Barreto <[email protected]> Co-authored-by: Dmytro Struk <[email protected]>
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 9, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Sep 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
agents
enhancement
experimental
Associated with an experimental feature
Ignite
kernel.core
kernel
Issues or pull requests impacting the core kernel
.NET
Issue or Pull requests regarding .NET code
PR: ready for review
All feedback addressed, ready for reviews
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Add streaming support to direct invocation of
OpenAIAssistantAgent
and alsoAgentChat
:Fixes: #4833
Fixes: #5643
Integrates with existing streaming support for
ChatCompletionAgent
.Description
Streaming returns the expected streaming content types as well as providing whole messages in the provided history.
Support both agent invocation (no-chat) and via
AgentChat
.Introduced:
StreamingAnnotationContent
StreamingFileReferenceContent
Added samples and tests.
Contribution Checklist